52 research outputs found
Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learning
Multitask deep learning has been applied to patient outcome prediction from
text, taking clinical notes as input and training deep neural networks with a
joint loss function of multiple tasks. However, the joint training scheme of
multitask learning suffers from inter-task interference, and diagnosis
prediction among the multiple tasks has the generalizability issue due to rare
diseases or unseen diagnoses. To solve these challenges, we propose a
hypernetwork-based approach that generates task-conditioned parameters and
coefficients of multitask prediction heads to learn task-specific prediction
and balance the multitask learning. We also incorporate semantic task
information to improves the generalizability of our task-conditioned multitask
model. Experiments on early and discharge notes extracted from the real-world
MIMIC database show our method can achieve better performance on multitask
patient outcome prediction than strong baselines in most cases. Besides, our
method can effectively handle the scenario with limited information and improve
zero-shot prediction on unseen diagnosis categories.Comment: EACL 202
BiERU: Bidirectional Emotional Recurrent Unit for Conversational Sentiment Analysis
Sentiment analysis in conversations has gained increasing attention in recent
years for the growing amount of applications it can serve, e.g., sentiment
analysis, recommender systems, and human-robot interaction. The main difference
between conversational sentiment analysis and single sentence sentiment
analysis is the existence of context information which may influence the
sentiment of an utterance in a dialogue. How to effectively encode contextual
information in dialogues, however, remains a challenge. Existing approaches
employ complicated deep learning structures to distinguish different parties in
a conversation and then model the context information. In this paper, we
propose a fast, compact and parameter-efficient party-ignorant framework named
bidirectional emotional recurrent unit for conversational sentiment analysis.
In our system, a generalized neural tensor block followed by a two-channel
classifier is designed to perform context compositionality and sentiment
classification, respectively. Extensive experiments on three standard datasets
demonstrate that our model outperforms the state of the art in most cases.Comment: 9 pages, 7 figure
Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networks
Mental health is a critical issue in modern society, and mental disorders
could sometimes turn to suicidal ideation without effective treatment. Early
detection of mental disorders and suicidal ideation from social content
provides a potential way for effective social intervention. However,
classifying suicidal ideation and other mental disorders is challenging as they
share similar patterns in language usage and sentimental polarity. This paper
enhances text representation with lexicon-based sentiment scores and latent
topics and proposes using relation networks to detect suicidal ideation and
mental disorders with related risk indicators. The relation module is further
equipped with the attention mechanism to prioritize more critical relational
features. Through experiments on three real-world datasets, our model
outperforms most of its counterparts
Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca
Foundational large language models (LLMs) can be instruction-tuned to develop
open-ended question-answering capability, facilitating applications such as the
creation of AI assistants. While such efforts are often carried out in a single
language, building on prior research, we empirically analyze cost-efficient
approaches of monolingual and multilingual tuning, shedding light on the
efficacy of LLMs in responding to queries across monolingual and multilingual
contexts. Our study employs the Alpaca dataset and machine translations of it
to form multilingual training data, which is then used to tune LLMs through
low-rank adaptation and full-parameter training. Comparisons reveal that
multilingual tuning is not crucial for an LLM's English performance, but is key
to its robustness in a multilingual environment. With a fixed budget, a
multilingual instruction-tuned model, merely trained on downsampled data, can
be as powerful as training monolingual models for each language. Our findings
serve as a guide for expanding language support through instruction tuning with
constrained computational resources.Comment: Work in progres
A Survey on Knowledge Graphs: Representation, Acquisition and Applications
Human knowledge provides a formal understanding of the world. Knowledge
graphs that represent structural relations between entities have become an
increasingly popular research direction towards cognition and human-level
intelligence. In this survey, we provide a comprehensive review of knowledge
graph covering overall research topics about 1) knowledge graph representation
learning, 2) knowledge acquisition and completion, 3) temporal knowledge graph,
and 4) knowledge-aware applications, and summarize recent breakthroughs and
perspective directions to facilitate future research. We propose a full-view
categorization and new taxonomies on these topics. Knowledge graph embedding is
organized from four aspects of representation space, scoring function, encoding
models, and auxiliary information. For knowledge acquisition, especially
knowledge graph completion, embedding methods, path inference, and logical rule
reasoning, are reviewed. We further explore several emerging topics, including
meta relational learning, commonsense reasoning, and temporal knowledge graphs.
To facilitate future research on knowledge graphs, we also provide a curated
collection of datasets and open-source libraries on different tasks. In the
end, we have a thorough outlook on several promising research directions
Automated Clinical Coding:What, Why, and Where We Are?
Clinical coding is the task of transforming medical information in a
patient's health records into structured codes so that they can be used for
statistical analysis. This is a cognitive and time-consuming task that follows
a standard process in order to achieve a high level of consistency. Clinical
coding could potentially be supported by an automated system to improve the
efficiency and accuracy of the process. We introduce the idea of automated
clinical coding and summarise its challenges from the perspective of Artificial
Intelligence (AI) and Natural Language Processing (NLP), based on the
literature, our project experience over the past two and half years (late 2019
- early 2022), and discussions with clinical coding experts in Scotland and the
UK. Our research reveals the gaps between the current deep learning-based
approach applied to clinical coding and the need for explainability and
consistency in real-world practice. Knowledge-based methods that represent and
reason the standard, explainable process of a task may need to be incorporated
into deep learning-based methods for clinical coding. Automated clinical coding
is a promising task for AI, despite the technical and organisational
challenges. Coders are needed to be involved in the development process. There
is much to achieve to develop and deploy an AI-based automated system to
support coding in the next five years and beyond.Comment: accepted for npj Digital Medicin
- …